After clawing their way back into the green, stocks fell again when Powell said the Fed's balance sheet will be "substantially smaller" than it is now, while also reiterating that the Fed can be patient about raising rates.
Powell's comments on the balance sheet came as the Treasury Department auctioned $16 billion in 30-year bonds to weak demand, which sent longer-dated bond yields to session highs.
The ratio of bids to the amount of 30-year bonds offered was 2.19, lower than the 2.31 at the 30-year auction in December.
The yield curve also steepened after minutes from the Fed's December meeting, released on Wednesday, showed that a range of policymakers said they could be patient about future interest rate increases and a few did not support the central bank's rate increase that month.
"It was definitely a dovish tone to it, which has aided the front end of the market," said Justin Lederer, an interest rate strategist at Cantor Fitzgerald in New York.
Benchmark 10-year notes gained 3/32 in price to yield 2.719 percent, down from 2.728 percent late Wednesday. The yield curve between two-year and 10-year notes steepened to 16 basis points, from 13 basis points early on Wednesday.
Two-year notes are the most sensitive to interest rate policy.